Human Decision Makers
Traditionally, the way decisions have been made has been to have a human expert assess the situation based upon their past experience and to make a judgment. This judgment then leads to an action which is described as a decision.
An advantage of this approach is that there already exist a number of experts who can be contracted to perform assessments—there is no need to create a new decision system. However, this advantage is counter balanced by several disadvantages—such experts may not exist in sufficient quantity and quality, may not be readily available when needed (24×7), may not be able to process information quickly enough, and may not make good decisions in situations where there are multiple uncertainties that interact in complex ways to affect the outcome of the situation.
To address the availability, repeatability, and uncertainty problems, computer-based decisions systems have been envisioned that would make decisions in lieu of experts, though perhaps with their input in building the decision system.
Expert Systems
Principal approaches to automating such decisions have been referred to as Artificial Intelligence, Expert Systems or Rule Based Systems. These systems consist of a number of conditional propositions (if X then Y or if X then Y with probability P) that compose a database of rules. When a transaction arrives, the values of its variables are then compared to the conditional propositions to reason to a likely outcome. Once an end state has been reasoned to, the system recommends a decision based upon that reasoning.
If a good rule base is in place, an automated decision can be made even at times when no human expert is available, and if the rules are superior to the judgment of a single individual it is possible that even better results will result—in any case a rule based system will have consistently repeatable results. For this system to be successful, it must have a good rules database. The process of deriving a rules database is usually called knowledge engineering. In this process, an expert in the creation of rules (called the knowledge engineer) is paired with a domain expert. The knowledge engineer then interviews the domain expert and tries to derive a set of rules.
In practice, this knowledge engineering process is often not very successful. Not only does it require access to a domain expert and to a knowledge engineer, but also it requires the domain expert to be articulate about the domain in terms that are meaningful to the knowledge engineer, and the knowledge engineer to be skillful at learning about the domain. Often this is not the case. Many experts are good at making judgments about situations but poor about explaining why they make those judgments. Because knowledge engineers themselves are not domain experts, they often encode rules specified by the experts, but the rules turn out to be overly simplistic.
A particularly frequent problem is that experts often explain their judgments after the fact by reference to obvious correlations between variables. For instance, an insurance products expert may observe that all the purchasers of maternity care insurance are women, and the knowledge engineer might encode this in an expert system as if gender=female then interest-in-maternity-care-insurance=high with probability P. A problem is that while gender has a high correlation with maternity care, it isn't a good discriminator. That is, while all people interested in maternity care insurance may be female, there may be many more females who are not interested in the insurance than who are. A better discriminator measure might include age, marital status, or enrollment in a Lamaze class. Quite often experts do not consciously recognize or cannot articulate these good discriminators, and the resulting automated models are often not very fast, and they are not very accurate.
Data-Derived Models
An alternative to the expert systems approach is to dispense with the expert and derive models directly from the data itself. In effect, this is equivalent to what the expert does when they build up experience over time—except that the automated system can build its expertise over far more transactions, and an automated system is capable of more reliable memory and more precise assessments of the degrees of correlation and discrimination among the variables. Therefore, in theory, data-derived models can be more accurate in their predictions than human experts are.
In practice, exhaustively determining all the cross-correlations can be computationally infeasible. Therefore a number of methods have been developed to “guess” at potential models, and apply heuristics to optimize them until they result in a sufficiently good, but probably suboptimal solution. Among these approaches are neural networks and genetic algorithms, as well as Bayesian networks. Some of these heuristic methods (but not including Bayesian networks) is that they generate “black-box” models. That is, it is fundamentally impossible to see what they have “learned”, or “why” they are producing certain recommendations.